Abstract In multi-hop inference models, path information is formed by fully mining and utilizing multi-step relationships between entities in the knowledge graph to accomplish knowledge inference. To solve the problems of sparse data and low reliability of inference paths in most of the existing sparse knowledge graph multi-hop inference models, a multi-hop inference model for knowledge graphs incorporating semantic information is proposed. Firstly, entities and relations in the knowledge graph are embedded into the vector space as the external environment for reinforcement learning training. Then, the semantic information of query relations and inference paths is employed to select the (relation, entity) pair with the highest similarity to expand the action space for path search by the agent, and thus the lack of sparse data in the inference process is compensated. Finally, the semantic similarity between the inference path and the query relation is utilized to evaluate the reliability of the inference path and it is fed back to the agent as a reward function. Experiments on several publicly available sparse datasets show that the inference performance of the proposed model is significantly improved.
Fund:National Natural Science Foundation of China(No.62062029,61762024)
Corresponding Authors:
DONG Rongsheng, professor. His research interests include knowledge graph, machine learning and computational thinking.
About author:: LI Fengying,Ph.D., professor. Her research interests include knowledge graph, machine learning and symbolic computing. HE Xiaodie, master student. Her research interests include machine learning and know-ledge graph.
[1] ROSSI A, BARBOSA D, FIRMANI D, et al. Knowledge Graph Embedding for Link Prediction: A Comparative Analysis. ACM Transactions on Knowledge Discovery from Data, 2021, 15(2). DOI: 10.1145/3424672. [2] 丁梦远,兰旭光,彭茹,等.机器推理的进展与展望.模式识别与人工智能, 2021, 34(1): 1-13. (DING M Y, LAN X G, PENG R, et al. Progress and Prospect of Machine Reasoning. Pattern Recognition and Artificial Intelligence, 2021, 34(1): 1-13.) [3] BOLLACKER K, TUFTS P, PIERCE T, et al. A Platform for Scalable, Collaborative, Structured Information Integration[C/OL]. [2022-04-16]. http://www.aaai.org/Papers/Workshops/2007/WS-07-14/WS07-14-004.pdf. [4] MILLER G A.WordNet: A Lexical Database for English. Communications of the ACM, 1995, 38(11): 39-41. [5] AUER S, BIZER C, KOBILAROV G, et al. DBpedia: A Nucleus for a Web of Open Data // Proc of the International Semantic Web Conference. Berlin, Germany: Springer, 2007: 722-735. [6] BORDES A, USUNIER N, GARCIA-DURÁN A, et al. Translating Embeddings for Modeling Multi-relational Data // Proc of the 26th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2013, II: 2787-2795. [7] WANG Z, ZHANG J W, FENG J L, et al. Knowledge Graph Embedding by Translating on Hyperplanes // Proc of the 28th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI, 2014: 1112-1119. [8] LIN Y K, LIU Z Y, SUN M S, et al. Learning Entity and Relation Embeddings for Knowledge Graph Completion // Proc of the 29th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI, 2015: 2181-2187. [9] JI G L, HE S Z, XU L H, et al. Knowledge Graph Embedding via Dynamic Mapping Matrix // Proc of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Stroudsburg, USA: ACL, 2015: 687-696. [10] YANG B S, YIH W T, HE X D, et al. Embedding Entities and Relations for Learning and Inference in Knowledge Bases[C/OL]. [2022-04-16]. https://arxiv.org/pdf/1412.6575.pdf. [11] NICKEL M, ROSASCO L, POGGIO T.Holographic Embeddings of Knowledge Graphs // Proc of the 30th AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI, 2016: 1955-1961. [12] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D Knowledge Graph Embeddings // Proc of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto, USA: AAAI, 2018: 1811-1818. [13] YANG F, YANG Z L, COHEN W W.Differentiable Learning of Logical Rules for Knowledge Base Reasoning // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 2316-2325. [14] XIONG W H, HOANG T, WANG W Y.DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2017: 564-573. [15] DAS R, DHULIAWALA S, ZAHEER M, et al. Go for a Walk and Arrive at the Answer: Reasoning over Paths in Knowledge Bases Using Reinforcement Learning[C/OL].[2022-04-16]. https://arxiv.org/pdf/1711.05851.pdf. [16] LIN X V, SOCHER R, XIONG C M.Multi-hop Knowledge Graph Reasoning with Reward Shaping // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2018: 3243-3253. [17] ZHOU X C, WANG P, LUO Q Q, et al. Multi-hop Knowledge Graph Reasoning Based on Hyperbolic Knowledge Graph Embe-dding and Reinforcement Learning // Proc of the 10th International Joint Conference on Knowledge Graphs. New York, USA: ACM, 2021. DOI: 10.1145/3502223.3502224. [18] ZHU A J, OUYANG D Q, LIANG S, et al. Step by Step: A Hie-rarchical Framework for Multi-hop Knowledge Graph Reasoning with Reinforcement Learning. Knowledge-Based Systems, 2022, 248. DOI: 10.1016/j.knosys.2022.108843. [19] LÜ X, CAO Y X, HOU L, et al. Is Multi-hop Reasoning Really Explainable? Towards Benchmarking Reasoning Interpretability // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2021: 8899-8911. [20] FU C, CHEN T, QU M, et al. Collaborative Policy Learning for Open Knowledge Graph Reasoning // Proc of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg,USA: ACL, 2019: 2672-2681. [21] LÜ X, HAN X, HOU L, et al. Dynamic Anticipation and Completion for Multi-hop Reasoning over Sparse Knowledge Graph // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2020: 5694-5703. [22] WILLIAMS R J.Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning, 1992, 8: 229-256. [23] ROCKTÄSCHEL T, RIEDEL S. End-to-End Differentiable Proving // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 3791-3803.
2022, Vol. 35 
